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ABSTRACT 

An extensive search of the literature and existing 
programs was undertaken to identify instruments that were being used 
to measure school effectiveness. Twenty-four instruments are 
currently available and are critiqued in this publication. Each 
critique reports the format and components of the instrument, its 
intended purpose and uses, reliability and validity information, 
distribution information, the supplemental materials available, and a 
description of its content. Four additional instruments in 
preparation are listed. This report also lists the districts 
each instrument, and related instruments. (BW) 
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ABSTRACT 

This review used two questions: "What is an effective school?" 
and "How is an effective school measured?" These questions led us to 
three types of literature^ atie first type was the "How to" 
literature — the numerous articles and reports on how to identify an 
effective school. That literature addressed issues and criticisms 
involved in defining an effective school - for example, the limitations 
of usinq achievement test scores as sole criteria for labeling one 
school effective and another ineffective. That type of discourse was 
unearthed chiefly through the ERIC system, in the form of journal 
articles and research reports. Since our primary purpose was to 
concentrate on extant instrum3nts developed from school effectiveness 
research, rather than with the process of labeling a school 
"ef fective*" , that related literature is not reviewed, but the articles 
and reports are listed in an annotated bibliography • 

The second type of literature we discovered when we asked about 
school effectiveness was the writing on identifying school-level or 
classroom-level process characteristics associated with effective 
schools. This literature addressed the process of discovering in- 
building fac;:ors associated with an effective school* This writinq 
most often took the form of technical reports or papers presented at 
professional conferences. Many instruments were developed and included 
for these purposes, serving as data collection devices for research 
studies. Since these were not our major interest either, the sources 
we found are also listed in an annotated bibliography. 

The third type of literature, (which is much more difficult to 
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discover) is the body of writing on instruments developed a posteriori 
to and based upon the school effectiveness research. This was the 
literature of greatest interest to us^ since our primary objective was 
to locate^ collect f describe and critique measures (i.e.^ 
questionnaires^ interview protocols^ and observation guides) that had 
items measuring the presence of previously identified characteristics 
of an effective classroom or school. These measures have been 
developed and used by state departments of education^ school districts, 
universities, research centers and educational laboratories. They are 
designed primarily for school self-assessment, and are usually embedded 
in a large*' school improvement program. 

Most of this third type of literature has not been submitted to 
the ERIC system. These measures were discovered by word of mouth. One 
source led us to another source. We asked of each, "Do you have an 
instrument that measures school effectiveness?" in almost all of the 
25-30 sources queried, they did. However, others* interpretations oE 
"school effectiveness instruments" differed from ours, resulting in 
stacks of a fourth type of literature, whatever these sources called 
"school effectiveness instruments." These instruments included 
descriptions of assessment programs (e.g., standardized and criterion- 
referenced or district-composed tests) and school climate measures. 
None of these were based directly and exclusively on the school 
ef fectivenesG research findings. Since we spent a great deal of time 
sorting through these, and because these do represent examples of what 
is considered by some as "school effectiveness instrumentation", they 
are listed in an annotated bibliography. 

During the collection process, we alternated in our thinking. 



First we believed that few of the kind of instruments we wanted did 

« 

exist* Then, after many telephone calls across the nation, we believed 
that numerous instruments existed. Finally, we now believe that few 
measures do exist and that they reflect various stages of development; 
some instruments are in stages of being piloted, some are being 
revised, some are being validated* We also believe that we have 
gathered the best and most frequently used ones, if not all of them. 

The school effectiveness instruments we found were reviewed using 
a structured critique format. This format was designed by asking our 
State Policy Task Force members what they would like to know about each 
instrument, as well as incorporating our own curiosities. The format 
includes information on reliability, validity, content, and structure. 
The instrument reviews are presented on the critique forms for the sake 
of uniformity and readability. 

The results of this project (reported herein) should make a direct 
contribution to both the researcher and the practitioner. The critique 
report represents the first c itical listing of what is actually 
available for school-level study of the presence of process variables 
identified by the research on school effectiveness. 

The report is organized into five sections: 

Section I Introduction 

Section II Effective Schools Measures 

Description and Critique Reports 
Section ill Annotated Bibliography of Related Instruments 
Section IV Bibliography of Related Literature on School 

Effectiveness 
Section V Instruments in Preparation 

Copies may be obtained from: 

Mid-continent Regional Educational Laboratory 
2600 S. Parker Rd., Bldg . 5, Suite 353 
Aurora, CO 80014 
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SECTION I 
Introduction 

This review was begun by asking two questions: •'What is an 
effective school?" and •'How is an effective school measured?" These 
questions led us to three types of literature. The first type is the 
"How to" literature — the r ^.merous articles and reports on how to 
identify an effective school. This literature addresses issues and 
criticisms involved in defining an effective school - for example, the 
limitations of using achievement test scores as sole criteria for 
labeling one school effective and another ineffective. Hiis type of 
discourse was unearthed chiefly through the EPIC system, in the form of 
journal articles and research reports. Since our primary purpose was 
to concentrate on extant instruments developed from school 
effectiveness research, rather than with the process of labeling a 
school "effective", this related literature is not reviewed, but the 
articles and reports are listed in Section IV of this report. 

Hie second type of literature we discovered when we asked about 
school effectiveness is the writing on identifying school-level or 
classroom-level process characteristics associated with effective 
schools. Itiis literature addresses the process of discovering in- 
building factors associated with an effective school. This writing 
most often takes the form of technical reports or papers presented at 
professional conferences. Many instruments were developed and included 
for these purposes, serving as data collection devices for research 
studies. Since these were not our major interest either, the sources 
we found are also listed in Section IV. 

Ttie third type of literature, (which J.s mc ch more difficult to 



discover) is the body of writing on instruments developed a posteriori 
to and based upon the school ef tectiveness research. This was the 
literature of greatest interest to us, since our primary objective was 
to locate, collect, describe and critique measures (i*e.> 
questionnaires, interview protocols, and observation guides) that had 
items measuring the presence of previously identified characteristics 
of an effective classroom or school. These measures have been 
developed and used by state departments of education, school districts, 
universities, research centers and educational laboratories. They are 
designed primarily for school self-assessment, and are usually embedded 
in a larger school improvement program. 

Most of this third type of literature has not been submitted to 
the ERIC system. Itieso measures were discovered by word of mouth. One 
source led us to another source. Vfe asked of each, "Do you have an 
instrument that measures school effectiveness?" In almost all of the 
25-30 sources queried, they did. However, others' interpretations of 
"school effectiveness instruments" differed from ours, resulting in 
stacks of a fourth type of literature, whatever these sources called 
"school effectiveness instruments." These instruments included 
descriptions of assessment programs (e.g., standardized and criterion- 
referenced or district-composed tests) and school climate measures. 
None of these were based directly and exclusively on the school 
effectiveness research findings. Since we spent a great deal of time 
sorting through these, and because these do represent examples of what 
is considered by some as "school effectiveness instrumentation", they 
are listed in an annotated bibliography included in Section III. 

IXiring this collection process, we alternated in our thinking. 
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First vfe believed that few of the kind of instrumentf. we wanted did 
exist* Then, after many telephone calls across the nation, we believed 
that numerous instruments existed. Finally, we now believe that few 

measures do exist and that they reflect various stages of developir t; 
some instruments are in stages of being p'^ ted, somi are being 
revised, some are being validated. We a^^so believe that we have 
gathered the best and most frequently used ones, if not all of them* 

The school effectiveness instruments we found were r'jviewed using 
a structured critique format. This format was designed by asking our 
State Policy Task Force members what they would like to know about each 
instrument, as well as incorporating our own curiosities. Trt^e format 
includes information on reliability, Vi»lidity, content, and structure. 
The instrument reviews are presented on the critique forms for the sake 
of uniformity and readability. 

Bie results of tbis project (reported herein) should niake a direct 
contribution to both tne researcher and the practitioner. The critique 
reports represent the first critical listing of what is actually 
available for school-level study of the presence of process variables 
identified by the research on school effectiveness. The majority of 
our contacts have requested a copy of this final report when completed. 
Our State Policy Task Force and local practitioners are also interested 
in adapting or adopcing the best of the lot (where possible) for LEA 
and SEA purposes. 

This report is organized into five sections: 



Section I 



Introduction 



Section II 



Effective Schools Measures 



Description and Critique Reports 
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Section III Annotated Bibliography of Related Instruments 
Section IV Bibliography of Related Literature on School 

Effectiveness 
Section V Instruments in Preparation 
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Section II 



EFFECTIVE SCHOOLS MEASURES 



DESCRIPTION AND CRITIQUE REPOF(TS 



I 
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EFFECTIVE SCHCXDLS MEASURES 
DESCRIPTION AND CRITIQUE REPORT 

1 • Instrument Title ; 

Building Level Effectiveness Survey/Classroom Level Effectiveness 
Survey (2 separate instruments) 

2. Authors: 

I 

Contact person: Ronald Smith, Goal-based Education Program, 
Research Associate, Northwest Regional Lab, 

3. Institution ; 

Northwest Regional Educational Laboratory, Portland, Oregon; 
contracted by Alaska Dept. of Education (assisted by individuals 
from cooperating institutions, i.e., SWRL, Center for School 
Improveaient, etc.). 

4 . Format and Components : 

Likert type questionnaires: Self-report Building Level 
questionnaire to be completed by "a principal, leadership ^am or 
whole school staff," Administration time - 1-1/2 hrs,: Cxaseroom 
level questionnaire completed by teachers » 30-45 min. 
Administration time. Could be used with either elementary or 
secondary, 

5. Intended Purpose and Uses : 

Not "Stand alone" instruments: embedded in a larger school 
improvement framework, NW Lab does not recommend instrument use 
out of context. Review of existing instructional program - no for 
individual teacher analysis. 

6 . Reliability Information ; 

No reliability study conducted. 

7 . Validation Information ; 

Assume cont.-^nt validity - no content validation completed or 
attempted, 

8 . Distribution Permission ; 

Draft Status: not for distribution: NW Regional Lab developed for 
Alaska State Dept. of Education urder contract. 

9 . Supplemental Materials ; 

Instructions for administration (2) 
Scoring guide (2) 

Rating Summary product-3 a rating graph for 13 or 17 subsections 
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NW Regional Lab 

Self-scorable by building level personnel 
1000 pages of training materials 

10, Content/Description (Effective Schools Measures) ; 

Both questionnaires tabulate responses on a frequency basis, in 
Likert format, itie response categories range from "Almost Never" 
to "Almost Always." Each of the two questionnaires can be 
completed by teachers; it is assumed that teachers will have 
completed the building level questionnaire prior to completing tha 
classroom level questionnaire. 

The Duilding level questionnaire contains 57 items in forced 
choice, Likert format; 13 categories or variables are represented 
according to NW's scoring instructions. The largest categories 
represented are: 1) Expectations, 2) Incentives and Recognition, 
and 3) Instructional issues (diagnosis, prescription, matching 
objectives and curriculum); use of time and monitoring staff 
performance also have relatively heavy coverage by items. The 
questionnaire does not seem to be slanted towards either elementary 
or secondary. Specific item # breakdown included. 

Hie classroom level effectiveness questionnaire contains 83 items 
in forced choice, Likert format. Many of the same items (phrased a 
bit differently) as the building level questionnaire are included. 
17 categories or "Clusters" are represented (according to NW's 
scoring instructions) , Fewer questions focus on expectations; 
largest clusters include 1) Objectives for instruction, and 2) 
Instructional procedures (strategies and resources) . Specific item 
# breakdown attached, 

1 1 . Comments (Effective Schools Measures) t 

Botli the response categories and the items contribute to specific 
problems with these questionnaires. Response categories are not 
all-inclusive. Usually,. Likert type responses range from "Never" 
to "Always"; these ranged from "Almost Never" to "Almost Always", 
thus excluding soiT.e possibilities, A^so, what is the difference 
between ^Rarely" and "Almost Never"? Response intervals are not 
evenly proportioned* 

Problems are also inherent in both the content and the structure of 
the items. First, all items are slanted in a positive manner 
(presumably for ease in scoring) which produces response bias. 
Second, the items do not necessarily reflect a direct 
correspondence to the findings from school effectiveness studies; 
i.e,, #47 on the classroom level (using advance organizers), #11 
(tips to parents with homework assignments) , etc. Although these 
may be considered by most to be good educational practice, there is 
no 1-1 correspondence to this body of research. Third, some items 
that are considered positive-#9, #10 - setting objectives to be met 
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NW Regional Lab 

by calendar d6te^ matching objectives to test items (shouldr't this 
be the other way around?) are not necessarily positive practices. 
The classroom level questionnaire did not address direct 
instruction frequency or varying group arrangements (small vs. 
A whole group) . Fourth^ the concepts or clusters are represented in 

several cases by a small number of items (i.e.^ 2 items for "Use of 
Assessment Data"). Fifth, some of the items are "double-barreled"? 
several questions are really checking two **ets of related, but 
different behaviors. 

Both instruments are easy to read, follow, and mark responses. 
Tallies and averages can be completed at the school level by 
district personnel^ Easy to score and graph. 
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Classroom Level Effectiveness Survey: 
Scoring Enstructicns 



There ace seventeen practice clustera in the Claaarooro Level 
Effectiveneaa gueationniare which will be uaed to aummarize the results : 



PRACTICE CXiUSTER 


ITEM NUNBERd 


AL X i£«no 


Learning Objectives 


f i—lQ 




Resoarcea m Materials 


All IT 

fix— 17 


/ 


Ins tr actional Strategies 


A 1 Q '^A 

ff lo-*^4 


7 
/ 


ftXpvCuaciona ror senavior 


V^D— Z / 




Routines and Procedures 


#28-34 


/ 


Clasarooin Behavior 


f oD— 41 


/ 


Expectations ror beaming 


§4^—4 J 


2 


Placement and Grouping 


«44--45 


2 


Stage Setting 


«46-4e 


2 


Instruction 


«49-56 


8 


Use of Time 


#57-64 


8 


Review and Re teaching 


«65-66 


2 


Student/Teacher Interaction 


«67-68 


2 


Incentives and Recognition 


#69-76 


8 


Asae 8 sme n t/ Al ig nme nt 


#77-78 


2 


Assessment/Procedures 


#79-80 


2 


Use of Assessnent Data 


#81-83 


2 



You will prepare one ratiiig for each cluster. Follow these steps. 

1. For all the combined items in a practice cluster, count the 

number of ratings in each of the possible response categories. 
For ex2unple, if ten people complete the survey, the responses 
for cluster on Expectations for Learning (3 items) may be 
distributed like thiu: 
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Building Level Effectiveness Survey: 
Scoring Instructions 

You will now compile responses and create a graph rating practice 
clusters covered in the Building Level Effectiveness Survey , There are 
thirteen practice clusters in the Building Level Effectiveness 
Questionnaire ^ listed below: 





XTiliM NUnn£U<o 




ijearning uDjecuiveB 


fx— 4 




Resources & Materials 


#5-9 


5 


Instructional Strategies 


#10-11 


2 


Expectations/Student Learning 


#12-13 


2 


Expectations/Social Behavior 


#14-21 


8 


Expectations/Staff 


#22-24 


3 


Use of Time 


#25-31 


7 


Incentives and Recognition 


#32-43 


12 


Parent Involvement 


#44-45 


2 


Assessmen t/Al ignment 


#46-47 


2 


Assessment/Procedures 


#48-49 


2 


Use of Assessment Data 


#50-51 


2 


Monitoring Staff Performance 


#52-57 


6 



These practice clusters will be used to summarize questionnaire results. 
You will prepare one rating for each cluster. Follow these steps: 

1« For all the combined items in a practice cluster, count the 

number of ratings in each of the possible response categories. 
For example^ if ten people complete the survey, the responses 
for the cluster on Expectations for Learning (3 items) may be 
distributed like this: 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

Instrument Title : 

School L>5arning Climate 
Assessment Instrument 

Authors ; 

W.B. Brookover 

Institution : 

Michigan state University 
Format and Components : 

52 item Likert-type forced-choice response questionnaire. 
Intended Purpose and Uses ; 

For school staff to assess the school learning climate. 

Reliability Information : 
None available at present. 
Validation Information : 

Validation to distinguish between high achieving schools and low 
ach ievi ng school s . 

Michigan and Tennessee - complete validation procedures and 
information to be sent when completed. 

Distribution Permission: 

- . . II ^ — — I 

••skeleton instrument" - will be revised to exclude items #1 and #18 
(no validation) . 

Supplemental Materials ; 
Scoring directions 



Michigan State u. 

10. Content/Description (Effective Schools Measures) : 

Likert-type questionnaire (52 items) . 
Addresses: 

1) Administrative Instructional Leadership (17 itemi.0 . 

2) Teacher Rewards (5 items) . 

3) Emphasis on Achievement (7 items) . 

4) Safe & Orderly Environment (10 itemR) , 

5) Expectations and Evaluations of Students (7 items) . 

6) Use of Test Data to E/aluate Instructional Program (8 items) • 

Items do seera to measure those categories on surface glance* 

Does not specify elementary or secondary: could perhaps be used at 
both levels • 



1 1 . Comments (Effective Schools Measures) ; 

Items are stated both positively and negatively. Response 
categories have even intervals. Measures behavior (latter section) 
and beliefs (first section) : 8 belief items; 44 behavior items. 
Directions for self-scoring at the school level. 

Large number of items per construct should yield greater 
reliability than other questionnaires. Questionnaire covers only 5 
areas. Excludes information about diagnostic-prescriptive approach 
to instruction, use of time (allocated, engaged or successfully 
engaged) and other relevant categories to effective schools. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

\ . Instrument Title : 

Delaware School Improvement Monitoring Process 

1) Staff Questionnaire 4) Teacher Interview 

2) Program and Building Observation 5) Principal Interview 

3) Support Staff Interview 6) Document Review 

2 . Author S ! 

Contact; Sid Collsin^ Delaware State Dept. of Education 

3. Institution : 

Delaware Department of Public Instruction 
Instruction Division - and - 
Delaware State Board of Education 

4. Format and Components : 

Observation and interview guides and questionnaire based upon 
Delaware's 11 program standards ^ most of which relate closely to 
the Effective Schools literature. RBS assessed in formulating 
these standards. 

5. Intended Purpose and Uses : 

For school self-assessment as bases for improvement plan, 

6. Reliability Information : 
None. 

7 . V alidation Information : 
None 

8 . Distribution Permission ; 
Limited. 

9 . Supplemental Materials : 

1) 16 monographs for teachers on tooics such as Parent Involvement, 
Better Discipline Ttirough Higher Expectat ions^ etc. 

2) Planning Handbook for School Improvement. 

3) Program Standards for Delaware schools. 

4) Delaware's Plan for Goal-Directed and Performance-Based 
Instruction. 

5) Flowchart of School Improvement Monitoring Process. 

6) Elements of Effective Instruction by Bill McCormick. 

7) List of Documents for Review. 
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Delaware 

10. Content/Description (Effect iv>e Schools Measures) : 

Principal (18 questions)^ T^achei (1 5 questions) , and support staff 
interviews (11 questions) are open-ended questions given to the 
interviewee prior to the interview. "Rxe teacher interviewee is 
asked to bring appropriate documentation to the inter vi.ew^ such as 
example 1 of retained students' work and lesson plans. Principals 
and support staff are not required to do this. Each question is 
identified as relative to one of the 11 standards. The 11 
standards are as followa: 

1. The school has a stated {.^ilosophy accompanied by school goals. 

2. The school has a planned program supported by staff, that 
follows the school's philogophy goals and identified priorities 
for improvement. 

3. The principal administers and supervises the school* s program. 

4. Itiere are written curricula that address the nine educational 
goals for Delaware and contain instructional objectives for all 
subjects . 

5. Pre/post assessment and analysis of each student's needs are 
integral parts of instruction. 

6. Learning activities are designed to accomplish the 
instructional objectives of each curriculum. 

7. Itie management of classrooms and other instructional areas 
facilitates learning. 

8. The school climate is conducive to learning and positive human 
interaction. 

9. Programs and services for meeting the special needs and 
interests of students are well-defined and coordinated with the 
school's instructional program. 

10. Ihere is a written promotion policy in operation which states 
performance criteria for assigning students to instructional 
levels and includes a plan for providing remedial instruction. 

11. Staff members are trained, assigned, and supervised. 
Ttiese standards were based on case studies conducted by the 

Delaware Dept. of Public Instruction on Effective Schools and the body 
of literature on effective schools. 

Itie staff questionnaire is administered in prepar;*.t ion for the 
monitoring visit. It contains 11 open-ended questions related to the 
11 standarc^s. Document review and building observation checklists are 
similarly constructed. 

11. Comments (Effective Schools Measures) ; 

There are several inherent weaknesses in the Delaware measures. 
First • the interviews seem to be very time-consuming to conduct with 
staff, particularly in asking for documentation. Also, some resistance 
could be anticipated on the part of teachers in complying with proving 
their statements by sharing lesson plans, retained students' work, etc. 
Tabulating open-ended responses to the questionnaire and compiling that 
information together with the teacher interviews (also open-ended) 
would be another extremely time-consuming task in the prvx:ess. 
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Delaware 

Finally^ there are relatively few questions per category (or 
standard) , These are very broad questions which may need 
additional probes to complete. 

Strengths of the process design include the monographs listing 
"Did you know that?" statements and verification from the research 
for educational practices. The Planning Handbook for School 
Improvement explicates the component development model for change* 
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EFFECTIVE SCHOOLS MEASURES • 
DESCRIPTION AND CRITIQUE REPORT 

Instiument Title ; 

The Connecticut Secondary School Development questionnaire^ and 
Secondary School Development Process student questionnaire^ and 
Student Achievement Profile 

Authors ; 

Alice Evans and William Gauthier^ Jr. 
Institution : 

Connecticut State Dept. of Education 
Contact: William Gauthier,, Jr, 

(In conjunction with the Northwest Regional Exchange) 
Format and Components ; 
3 components; 

Two Questionnaires and Achievement Profile 
Three forms of Student Questionnaire 

Two forms of Secondary School Development Questionnaire 
Intended Purpose and Uses; 



School Assessment; basis for improvement plan 
Reliability Information : 

Plans for internal consistency Reliability^ Stability Reliability. 
Has only been in use for six months. 

Validation Information ; 

None Enclosed. 

Distribution Permission ; 

Research Edition^ copyright 1983. Not to be reproduced without 
permission. 

Supplemental Materials ; 

(See Elementary Edition Critique) 
Student Achievement Profile 
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Connecticut S.D.E, 

Content/Description (Effective Schools Measures) : 

Bie Student Achievement Profile (separate from Instruments) is 
baaed on Edmonds' definition of an effective school — one in which 
the proportion of low income students attaining the minimum mastery 
of basic skills is the same as the proportion or other students' 
attaining minimum mastery. The Achievement Profile gathers three 
types of information: 

!• A measure of achievement in reading, language arts, and 
mathematics for each student (standardized test results), 

2. An index of social class for each student (free lunch count , 
parents' occupation or education) • 

3, A criterion designation for minimum mastery (school staff make 
this decision - example uses 30th percentile criterion level). 

Data are plotted on horizontal axis (percentile rank by 10 points 
intervals) by income level (high, average, low) on the vertical 
axis. Achievement data for each of the basic skills are displayed 
for each grade level tested. 

Each form of the student questionnaire (3 forms) contains 50 
items. No description accompanied the questionnaire. A Likert 
scale format was used with response categories ranging from 
"Strongly Agree" to "Strongly Disagree." Some negatively stated 
items included. Several items don't appear to be directly related 
to the Effective Schools Research (e.g., #3, 9, 15, 16, 50 - Form 
1) . (Reviewed 2 different Form I's) . 

The secondary school development questionnaire contains 100 
items - same format * intended for staff. Two forms exist. 

1 1 . Comments (Effective Schools Measures) : 

No enclosed written information concerning the school level student 
questionnaires. Data displays show bar graphs and percent of 
responses in each response category for each item. 

T^is series will need updated information on reliability and 
validity when completed and available from the Connecticut State 
Department. 
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EFFECTIVE SCHCX)LS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 • Instrument Title : 

Itie Connecticut School Effectiveness Interview and Questionnaire 
(elementary) 

Secondary School Development Questionnaire^ Student Questionnaire. 

2. Authors : 

Elementary : Robert Villanova and William Gauthier, C. Patrick 
Proctor^ Joan Shoemoker^ Herman Sirois^ Martin Gotowala. 
Secondary : William Gauthier and Alice Evans. 

3. Institution : 

Connecticut State Department of Education (in conjunction with the 
Northwest Regional Exchange) . 

4 . Format and Components : 

Elementary and Secondary Multiple Methods of Data Collection. 

5. Intended Purpose and Uses : 

For school self-analysis: as basis for an action plan for school 
improvement^ formed by a task force from the building and 
facilitated by data collectors. 

6. Reliability Information : 

Some construct scales have loii/ inter-rater reliabilities and are in 
need of farther revision. Generally high inter-rater reliabilities 
for interview category classifications. Will be repiloted^ Spring 
'82 - plans for internal consistency reliability^ stability and 
factor analysis of questionnaire. 

7 . Validation Information : 

Interview content validity assessed by panel of experts who sorted 
items into categories; recommended exclusion or modification; then 
piloted on 6 teachers and administrators. Questionnaire content 
validity established by 1) judges' agreement and 2) independent 
classification of items to categories. Piloted on 6 teachers. 

8 . Distribution Permission : 

Research Edition - not to be reproduced without permission. 
Copyright ad. 

Send for Robert Villanova' s dissertation establishing reliability 
and validity. 
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Connecticut 

9. Supplemental Materials : 

1. Monograph: Instructionally Effective Schools: A Model and a 

Process 

2. Elementary Interview (45 min. Administration Time) 

3. Elementary questionnaire 

4. Secondary faculty questionnaire 

5. Secondary student questionnaire 

6. Elementary data display 

7. Secondary data display 

10- Content/Description (Effective Schools Measures) ; 

Half a schools' faculty are interviewed using Connecticut 
School Effectiveness interview (elementary^ 67 items) probing 7 
school effectiveness characteristics. Itiese are: 

1. Safe and orderly environment (5 items) 

2. Clear school mission (11 items) 

3. Instructional Leadership (14 items) 

4. High expectations (10 items) 

5. Opportunity to learn and student time-on-task (9 items) 

6. Frequent monitoring of pupil progress (8 items) 

7. Home-school relations (10 items) 

In other half of schools, faculty respond to the Connecticut 
School Effectiveness Questionnaire (100 items) measuring the 7 
characteristics. Ohe interview is highly structured with 
directions to interviewers (who are trained) . The questions are 
open-ended at the conclusion, asking interviewee to identify the 
schools' strengths and weaknesses. A forced choice set of 
responses (ranging from extremely positive to extremely negative) 
is used by the interviewer to classify subjects' responses. 
Comments are also recorded if response includes other information. 
Items focus specifically on reading and math instructional conduct 
in Category Ii. Many of the items are very similar in content to 
Brookover's instrument. A scoring guide for the interviewer allows 
ranking of response on a 1-5 basis. 

T^e elementary questionnaire is a 100 item Likert scale with 
response categories ranging from "Strongly Disagree" to "Strongly 
Agree." One in every 7 items is reverse scored (i.e., stated 
negatively). Categorical items areyiot grouped together, but 
scattered randomly throughout the questionnaire. The items appear 
to have a strong correspondence to the school effectiveness 
research at an appropriate level of generality. 
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Connecticut 

1 1 . ComiDents (Effective Schools Meaaurea) : 

Based on a modification of the New York School Improvement Project ♦ 
Also used to gather an Achievement Profile by gathering student 
achievement data by student economic class subset and archival data 
(report cards, student handbooks, mastery skill checklists) • Input 
from Michael obhen was used who suggested interview and observation 
techniques^ 

These instruments are a portion of a multi-trait, multi-method 
assessment process. Plans for technical manual of sophisticated 
validation and reliability procedures from Robert Villanova's 
dissertation have been developed. 
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/-A*, STATE OF CONNECTICUT 

fl^f^^ lUrAHTMi\I Oh t:iH(ATl()\ 

Reliability Estimates for the Factor Categories 
on Connecticut's School Effectiveness Questionnaire 

Number Alpha Internal 

Factor Categories* of Items** Consistency Reliability 

Safe and Orderly 13 .90 

Leadership 23 .94 

Clear School Mission 11 .84 

Home/School Relations 12 .86 



♦Factor Method: orthogonal solution-varimax. 

♦Factor names were devised from judgemental interpretations of the loadings. 

**Number of items reflect items that loaded .35 and better. 

Note: this is a preliminary analyses, further analyses will be forthcoming. 
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Reliability Estimates for the Judgemental Categories 
Connecticut School Effectiveness Questionnaire 



Categories 



Number Alpha Internal Test-Retest* 

of items Consistency Reliabilities Reliabilities 



Safe and Orderly 
Environment 



15 



.87 



.85 



Clear School Mission 



14 



Instructional Leadership 25 



Expectations 



Opportunity to Learn 



Monitoring Student 
Progress 



12 



12 



12 



.90 
.93 
.55 
.66 
.77 



.90 
.83 
.69 
.74 
.67 



Home/School Relations 



10 



.89 



.82 



Total N=423 

*Test/Retest reliability was generated using only one school. 
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Reliability Estimates for the Judgemental Categories on 
the Connecticut School Effectiveness Interview 



Categories 



Number 
of Items 



Alpha Internal 
Consistency Reliability 



Safe and Orderly Environment 



Clear School Mission 



Instructional Leadership 



Expectations 



Opportunity to learn 



Monitoring Student Progress 



Home/School Relations 



5 

11 
14 
10 
9 
8 
10 



.66 
.93 
.81 
.69 
.78 
.86 
.66 



Total N=^?3 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instrument Title : 

(8 questionnaires) 

1) Pupil Information Profile 

2) Pupil Statistical Questionnaire (to be completed by the 
teacher) . 

3) The Teacher Questionnaire 

4) The Teacher Statistical Questionnaire 

5) T he Principal Statistical Questionnaire 

7) Teacher Aide Statistical Questionnaire 

8) School Site Survey 

2. Authors : 

Contacts Gilbert R. Austin 

University of Maryland 

3. Institution ; 

Maryland State Dept. of Education 
Maryland University of Baltimore 

4 • Format and Components : 

Statistical Questionnaires collect background data; e.g.^ age^ 
years experience^ number and kind of degrees held; other 
questionnaires measure attitudes; school site survey collected 
factual information - physical plant, resources, curriculum, PTA, 
etc. 

5. Intended Purpose and Uses : 

Designed for process evaluation to distinguish differences in 
process characteristics between high achieving and low achieving 
schools - elementary level - (exploratory) . 

6. Reliability Information ; 

Student and Teacher questionnaires 
Test-Retest (stability) Reliability 
(detailed technical report not included) 

7 . Validation Information : 
None available. 

8 . Distribution Permission : 
Contact University of Maryland 



ERLC 



29 



25 

'toryland S.D.E, 
Maryland U* -Baltimore 

9. Supplemental Materials : 

1) ERIC document EDI 60 644 "Process Evaluation: A Comprehensive 
Study of Authors" 

2) "Characteristics of Effective Schools", Gil Austin and Phyllis 
Utterback 

3) Executive Summary 

4) "Exemplary Schools and tneir Identification", New Directions 
for Testing and Measurement , 10 , 1981, pp. 31-41, 

10. Content/Description (Effective Schools Measures) ; 

Ttie items on all the questionnaires fell into two categories: 1) 
those items to v*)ich a subject responded on a bi-polar Likert-type 
scale (some stated positively, some negatively) vAiich produced 
ordinal data; 2) those to which subjects responded by choosing a 
discrete category, producing nominal data. Many of the questions 
were taken from studies done under James Coleman by the U.S. Office 
of Education, Ralph Tyler, John Flanagan in Project Talent, the 
California Study of Educational Progress, the New York Study on 
Production Functions and a locally developed student question- 
naire !• 

1 1 . Comments (Effective Schools Measures) ; 

T^iese questionnaires (and observation instruments part II) were 
developed (or used) to identify distinguishing factors or process 
characteristics associated with identified high and low achieving 
schools. These were exploratory in nature and not developed to be 
used as a self-assessment tool for a school to compare itself on 
the variables associated with effective schools. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTIOr AND CRITIQUE REPORT 

1 . Instrument Title : 

Reciprocal Category - System (RCS) , and 
Teacher Practice Observation Record (TPOR) 

2» Authors: 

Contact: Gil Austin 

University of Maryland 

3, Institution : 

Maryland State Dept. of Education 
University of Maryland 

4 • Format and Components ; 

RCS is a modification of the Flanders Verbal Interaction System 
(Student-teacher Verbal Interaction) . 

TPOR is an instrument for systematically observing teachers' 
behavior « 

5. Intended Purpose and Uses : 

Guide for classroom observation as basis to distinguish systematic 
differences in processes between high and low achieving classrooms. 
Used to aggregate data at the school level not classroom level. 

6. Reliability Information ; 

Feasible to train observers with acceptable reliability - however ^ 
reliability coefficients not reported. 

7. Validation Information : 

None provided - used extant instruments developed by external 
researchers. 

8 • Distr ibut ion Permissio n; 

Contact authors of RCS and TPOR. 
9. Supplemental Materials ; 

None 
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Maryland S*D*E., U* of Maryland 

10. Content/Deacr Iptlon (Effective Schools Measures) ; 

The Reciprocal Category System (RCS) is a modification of the 
Flanders; It was developed by' »r • Ober of the University of South 
Florida, An equal number of categories are assigned for teachers' and 
students* talk. The RCS Sydtem looks only at talk and classifies all 
talk Into 10 categories of 1) norms (informallzes) the climate^ 2) 
accepts^ 3) amplifies the contributions of another, 4) elicits, 5) 
responds, 6) initiates, 7) ilrects, 8) corrects, 9) cools (formalizes) 
the climate, and 10) silence or confusion. This information is 
developed into a score that relates to a score on a verbal continuum 
from one extreme to another, e.g., •'warm'' to •'cool." Requires 2-2 mln. 
observations. 

The Teacher Practices Observation Record (TPOR) records teacher 
behavior in 7 categories: 1) nature of the situation, 2) nature of the 
problem, 3) development of ideas, 4) use of subject matter, 5) 
evaluation, 6) differentiation, 7) motivation control. Scores are 
developed in each category in addition to a total score which 
summarizes the degree of chlld-centeredness for the school. Requires 
3-3 minute separate observations within a 20 minute period. The TPOR 
was also constructed to indicate experimental teaching methods versus a 
more traditional or conservative approach. 

1 1 . Comments (Effective Schools Measures) t 

These classroom observation systems or instruments were used in 
conjunction with and supplemental to the 8 questionnaires. The 
Maryland Study found few distinguishable differences between high and 
low achieving schools from the results of the TPOR. The RCS did reveal 
great variation, e.g., teachers in high achieving schools gave students 
more direction, elicited more comments from students and corrected 
students more frequently than in low achieving schools. Judgements as 
to how reliable or valid these Instruments are cannot be made based on 
the information provided. The inclusion of classroom observation 
(another multi-method, multi-trait approach) lends further information 
but does not triangulate or substantiate the process variables 
identified by the questionnaires. These observation systems are not 
developed based upon the effective schools research per se, and were 
used as exploratory measures to Identify characteristics of an 
effective school. 
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EFFECTIVE SCHOOLS MEASURES • 
DESCRIPTION AND CRITIQUE REPORT 

Instrument Title ; 

Indicators of Quality Schools 

Instrument to Assess the Educational Quality of Your School 
Authors : 

VIn. Dean^ T. Howard^ K. DePew^ R. Ewy^ J. Harrigan^ J. Hennes^ T. 
Knauer, Sr. A. lipomas, G. Villars. 

Institution ; 

Colorado State Department of Education^ Denver^ Colorado. 
Contact: Eugene Howard 

Format and Components : 

One questionnaire (42 items ^ 203 sub-items) to be used K-12. 
Respondents may be staff members^ "informed pupils or parents." 
One hour administration time. 

Intended Purpose and Uses ; 

Designed to assist a school in identifying areas of strength and 
areas for improvement in school quality: K-1 2 use. To be used in 
a school self-study. 

Reliability Information ; 

No psychometric data attempted or reported. 
Validation Information : 

No psychometric data attempted or reported. 
Distribution Permission ; 

Available from Colorado State Dept. of Education at cost. 
Supplemental Materials : 

1. Overview 

2. User's Guide {including scoring and summary plot) 

3. Review of the literature on ef'.ective schools 



33 



29 

Oolorado S.D.E. 

10. Content/Description (Effective Schools Measures) : 

Itie assessment items are based upon the effective schools 
literature as well as (according to the User*s Guide) •'agreed-upon good 
school practice." (p.1) Respondents code their answers to 42 items 
(203 sub-items) in 2 categories, "What is?" and "What Impact 
Potential?" Some of these items are repeated since some activities will 
have impact on more than one quality indicator. 

Ihe questionnaire is organized around three dimensions: I) Stuuont 
outcomes: achievement 6 satisfaction, li) School Leadership, 
Instructional and Institutional Characteristics, III) Accountability 
Accreditation/Planning Process. Twelve sub-categories are directly 
assessed: 1-1) curricular (13 items; congruence, 2) assessment (17 
items), II-3) Principal Leadership (18 items), 4) high expectations (13 
items, 5) school-wide norms (9 items), 6) school climate (39 items), 7) 
monitoring and feedback (11 items), 8) time-on-task (11 items), 9) 
organization and management (23 items), 10) instructional effectiveness 
(18 items), Ill-ll) parent and community involvement (14 items), 12) 
accountability, accreditation, planning (18 items)* Can be sub-scored 
in each area. 

The questionnaire measures beliefs about behavior and attitudes 
toward that behavior. Respondents rank order frequency of present 
occurrence and impact potential if that activity were present in the 
school. 

1 1 . Comments (Effective Schools Measures) : 

The Quality Schools Indicators , as is typical with school 
effectiveness instruments, has several major flaws. First, the 
instrument l.^cks reliability data or validation data. No psychometric 
studies have been conducted. Second, not all items are based on school 
effectiveness literature, but are thought to reflect "good educational 
practice." Third, bias in responses is encouraged by 1) the 
introductory paragraphs executing what positive conditions are and 2) 
all items are slanted positively. Fourth, several of the items are 
double-barreled, measuring two or three separate behaviors. Fifth, 
some items are more appropriate for one type of respondent than 
another, e.g., "I like wrking in this school" — very few parents would 
be able to answer that item. 

It is interesting to note that twice as many items (39) measure 
school climate as measure any other construct or category. Generally, 
there are a large number of items per category (unusually large in 
comparison to other questionnaires), which should increase reliability 
coefficients in internal consistency. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 • Instrument Title ; 

Elementary Program Review Handbook (84 pgs.) 
Secondary Program Review Handbook (9 3 pgs.) 

2. Authors : 

California State Dept. of Education 
Contact: Walter Denhara and/or Dr. Doornek 

3, Institution : 

California State Department of Education (Developed over a 10-yr, 
period; redeveloped in 78-79-80) • 

4» Format and Components : 

1) Directions to evaluators and for final report 

2) Observation, interview and document guides 
(elementary and secondary editions for 80-81) 

5, Intended Purpose and Uses : 

Evaluation criteria established to guide members of a program 
review team (internal or external) to judge effects of the school 
program and opportunities for improvement - can be used for peer 
observation on individual classroom level* 

6, Reliability Information ; 
None 

7, Validation Information ; 
None 

8, Distribution Permission : 
Contact State Department* 

9 , Supplemental Materials : 

!• Elementary School working papers (quality document and 

compliance document) • 
2, Secondary school working papers (quality document and 

compliance document) • 
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California State Department of Bducation 

10. Content/Description (Effective Schools Measures) : 

Bach guide lists the area of investigation (e.g., student 
activities), the primary source of information (e.g., school or 
classroom observation and/or interviews) and verifying sources of 
information (e.g., interviews, documents). The guide is quite 
comprehensive and directive. An example from the guide for collecting 
information about how students are learning (elementary level) includes 
guiding questions such as, "To what extent is direct instruction a part 
of the students* daily instruction?" and "Does the student understand 
what he or she is expected to do^ how to do it^ and why?" (p. 11-10) . 

Hhe quality documents and compliance documents focus on 
opportunities for improvement, e.g., "How Students Are Learning", "What 
Students Are Learning", "The Environment in Which Learning Takes 
Place", "Improving Staff Development", and "Improving District 
Support". These consist of one or two page lists of general 
suggestions, such as (under "Improving Staff Development") "h. allocate 
sufficient time for staff development activities", "d« set priorities 
for group and individual staff development." Many of these suggestions 
are duplicated between elementary and secondary levels. Its primary 
purpose is program evaluation, not building assessment. 

1 1 . Comments (Effective Schools Measures ) : 

In a definitive sense, this is not an instrument based solely on 
the effective schools research developed as an assessment device for 
measuring the presence of those characteristics. The criterion 
standards are bulky, extensive and represent a program or package 
rather than a tool. Although excerpts could be used (particular 
potential for peer observation using 1 criteria at classroom level) for 
initial looks at a schools* program, the entire package of guides is so 
exhaustive as to prohibit external use to measure school effectiveness 
in other states* distiricts. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

Instrument Title ; 

2nd Edition, New York City School Improvement Project Assessment 
Instruments: 1)* Principal Interview Form , 2)* Asst. Principal 
Interview Form , 3)* Classroom Instructional Staff Interview Form A 
*__B, 5) * Classroom Teacher Questionnaire , 6) Special Program 
Instructional Staff interview Form , 7) Special Program Teacher 
Qu estionnaire , 8) Para Interview Form , 9) Auxiliary Staff Interview 
form , 10) Parent Interview Form , 11) Building and Grounds 
Observational Assessment . 

^■Reviewed 

Authors : 

1980, Board of Education of the City of New York* 
Institution ; 

New York City Public Schools 
Contact! Anthony J, Alvardo 

Format and Components ; 

11 components including questionnaires, interview guides (2 forms 
each) and observation instrument. 

Intended Purpose and Uses ; 

To be used as a needs assessment system for school improvament 
effort. 

Reliability Information ; 
None 

Validation Information ; 
Unknown 

Distribution Permission ; 

Must submit written request to Chancellor for permission to 
repr int ; must acknowledge source. 

Supplemental Materials ; 

NOTE ; Supplemental information is still pending from NY City 
Public Schools. 
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New York City Public Schcx>ls 



10» Content/Description (Effective Schools Measures) ; 

The Classroom Teacher Questionnaire is 13 pages in length. 
Teachers are queried using a combination of forced-choice and 
open-ended response formats. The items include queries concerning 
reading and math methods and approaches, goals and objectives, 
coordination of instruction, observations, lesson plans, 
communication, supervision, inservice resources, parent-teacher 
relations, admin. -teacher relations, expectations, climate testing 
and progress reports. Questionnaire is self-administered, 
containing approximately 4-6 questions per topic. 

The Classroom Instructional Staff Interview is 9 pages in length. 
The questions are all open-ended (some require a yes-no response 
but ask for elaboration) . The questions are almost exactly the 
seune as the teacher questionnaires, but phrased differently. 

The Principal Interview Form consists of 8 pages of open-ended 
questions. The items cover topics such as supervisory and 
administrative methods, perceptions of effectiveness with staff and 
parents, communication, goals and objectives, expectations, 
school-wide curriculum, reading, math and writing approaches, basic 
skills instruction resources and programs, mandated time for 
coordination of instruction, innovative instruction encouragement, 
staff characteristics, student behavior, school climate (i.e., 
safety), parent support, testing, diagnosis, teaching by objective, 
individualized instruction and school strengths and weaknesses. 
Few questions (1-3) measure eacV« category. 

The Assistant Principal Interview Form asks almost all the same 
questions (sometimes phrased differently) as the Principal 
Interview. 



1 !• Comments (Effective Schools Measures) ; 

Tliese 4 instruments appear to be based directly upon the school 
effectiveness research. Hie open-ended response formats lead to 
difficulties, as well as strengths, in several ways. First, data 
analysis would be cumbersome, requiring tabulation of lengthy 
responses. Second, this type of data does not produce neat bar 
graphs or scoring ease to show discrepancies or agreement between 
and among respondents. Third, no information is available on 
construct validation. Reliability and validity studies were most 
likely never conducted, due to format. 

Ttie greatest strengths of the teacher and principal assessment 
forms are: 1) the content of the questions, and 2) the 
tr iangulation provided hy asking similar or exact same questions of 
each type of respondent. Many of the questions are designed to 
"double check" or provide a kind of reflexivity concerning what 
actually occurs within a ' jhool by matching self-perceptions with 
others • perceptions. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 . Inatruroent Title ; 

New Jersey Education Association School Effectiveness Training 
Prograun: 1) Teacher Questionnaire, 2) Student Survey, 3) SET 
Institute Evalulntion, 4) NEA Teacher Needs Assessment Survey. 

2. Authors ; 

» 

Produced by NJEA Instruction 
Contact: Norman Goldman, NJEA 

3. Institution ; 

New Jersey Education Association 

in collaboration witb Research for Better Schools, Inc., 1981. 

4 . Format and Components ; 

Inservice Education Handout Packet and embedded 60 item 
questionnaire (NEA Teacher Needs Assessment Survey) . 

5. Intended Purpose and Uses ; 

Brief Needs Assessment as basis for Action planning in Inservice 
program used on school by school basis. 

6. Reliability Information; 



7 . Validation Information ; 
Piloted in 1978. 

8. Distribution Permission : 

Contact New Jersey Education Assoc. or RBS. 

9. Supplemental Materials ; 

1) SET School Contact Form 

2) SET School Site Report Form 

3) Staff Development Planning Wbrksheet 

4) Action Planning 

5) Factors in Problem-solving 

6) A Problem-solving Program for Defining a Problem and Planning 
Action 
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New Jersey Education Assoc. 



10. Content/Description (Effective Schools Measures) : 

The NBA Instructional Staff Questionnaire requests 2 responses to 
each of the 60 items • One is "what should be" - with response 
categories ranging from "never" to "always". The other response 
required is "what is". 

This is one of 4 or 5 extant instruments which have been updated. 
The new information and questionnaires are currently being 
forwarded to McREL. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 



2. 



3. 



4« 



5* 



6. 



7. 



8. 



Instrument Title ; 

Profile of School Excellence 

Assessment Package: 7 different data sources 

Protocols and questionnaires 

Authors ; 

Jack Sanders 

AEL 

1983 

Institution ; 

Appalachian Educational Laboratory 
Format and Components ; 

Assesses school effectiveness variables ^ using protocols and 
questionnaires designed for teachers, principals, students. 
Superintendent interview, central office staff. 

Intended Purpose and Uses ; 

P3r diagnosis at district and school level: used as part of AEL's 
assessment/prescription service* 

Reliability Information ; 

Average interrater reliability .92. 

Validation Information ; 

Validated by expert judgement - authority model pilot-tested. 
Discriminated among high and low-achieving schools. 

Distribution Permission ; 

Will not release for examination? private non-NIE product. 
Purchase only. Corporation. Copyrighted. McREL does not have 
this inst rumen tat ion. 

Sup plemental Materials : 

1) Data form 

2) Graphic profiles (bar graph, perceptions) 

3) Report to superintendent 

a. vignette on each school 

b. recommendations to district 
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Appalachian Educational Laboratory 

10* Content/Description (Effective Schools Measures) ; 



ftjestionnaires ar^ administered on a random basis to 10% of 
districts' students, Grades 7-12 and 50% of teachers at all grade 
levels^ Each participating principal is interviewed on site* 



1 1 • Comments (Effective Schools Measures) : 

No comments can be made since actual instrumentation was not shared 
with McREL. Information in critique report came f-'ora oral 
telephone conversation and 1-page abstract. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

Instrument Title : 

School Improvement Questionnaire (SIQ) 
Authors : 

Bob Ewy^ Larry Hutchins^ Susan Everson, Ann Riley 
I nstitution : 

Mid-continent Regional Educational Laboratory 
Format and Components : 

Questionnaire - consisting of 73 items answered on a Likert scale: 
always true^ usually true^ sometimes true, never true, don't know. 
Responses are recorded on a sens?^-mark score sheet which is 
mach i ne-scor ed « 

Intended Purpose and Uses ; 

To measure the perceptions of faculties and principals concerning 
their engagement in behaviors or activities which are associated 
with effective schools. 

Reliability Information ; 

None to date - however, data are being collected for a test-retest 
reliability study and an estimation of internal consistency o£ the 
category scores. 

Validation Information: 



None to date - A discriminate validity study and a factor analysis 
are planned. 

Distribution Permission : 

May not be used unless purchased from McREL. 

Supplemental Materials ; 

A technical manual is being written. 
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McREL 

10. Content/Description (Effective Schools Measures) ; 

Ttie 73 items (94 on the principal/administrator version) are scored 
in the following 15 improvement areas: 

t of items 

73 94 



10 Diagnosing student needs 

The items in this area refer to the 
activities that teachers^ principals^ 
administrators undertake to determine what 
the students need to learn^ and what the 
appropriate instructional techniques ^re for 
teachinn them, 

9 Prescr ibing instructional outcomes 

Ihe items in this area refer to the 
explication and communication of 
instructional objectives and expected 
outcomes by teachers to students, district 
to teachers and school to community. 

8 Communicating expectations of success 

Hiis area comprises items that indicate 
that teachers and administrators communicate 
the belief that all students can and will 
succeed. 

7 Giving student responsibility for 

independent learning 

In thi-i area, items are included that 
describe the ways in which students and 
teachers are given autonomy and 
responsibility for their own activities. 

10 Providing effective, interactive instruction 

The items in this area provide 
information about the extent to which 
faculty and staff use effective teaching 
strategies and are supported in this effort 
by administrator s . 

9 Providing time for academic activities 

This area contains items that assess 
whether students and teachers are given 
sufficient time to complete tasks, how well 
they use that time, and if they are 
successful on the tasks they complete. 

11 Providing knowledge of outcomes and feedback 

In this area, the items refer to the 
ways in which progress toward goals is 
monitored and communicated to the students, 
teachers, district, and community. 
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^ 6 Providing success experiences^ rewards^ and 

praise 

The Items In this area describe the way 
in vrtiich students* and teachers' accomplish- 
ments are acknowledged and encouraged. 

9 16 Creating a positive school climate 

This area includes it^ms that determine 
how the teachers and administrators are 
involved in creating a school environment 
that la safe^ clean^ predictable^ pleasant^ 
and conducive to learning • 

9 9 Creating cooperation and collegiality among 

faculty and staff 

These items refer to the way«^ in %*ich 
personal concern and support are provided to 
teachers by each other and by 
administrators. 

6 6 Soliciting parent il support 

Hie items in this aiea describe the 
ways that the school encruiages and develops 
good school-parent and school-community 
rela ions* 

25 25 Teachers 

OJiis area contains all items that refer 
to activities and behaviors conducted by 
teachers across all the first eleven areas* 

37 59 Environment 

Ihis area contains all items that refer 
to resources (physical, educational, and 
psychological) and policies that develop and 
support the other areas. 

II 11 Leadership 

TSiis area contains all items that refer 
to activities and policies that take place 
at the school level across the other areas. 

11 11 Principals 

•niis area contains all the items that 
refer specifically to activities and 
behaviors in which the principal engages 
across the other areas. 



The SIQ is generally completed by all the faculty members in the 
schools that are involved in training. Results of the SIQ are reported 
in two forms for each Improvement area. One is the comparison of the 
teachers' and principals* average scores* Ihis is presented in a cjraph 
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NcREL 

form to make it easier to understand. Ttie second is the distribution 
of responses^ average score, and disagreement aunong respondents for 
each item on the SIS. Team members from schools participating in 
morel's Effective Schools Program are given instruction in interpreting 
these results and incorporating them into their school improvement 
plans. The SIS is generally administered before the training for all 
faculty in participating schools, and after the training for team 
members. In some cases, however, the post-measure has been taken with 
non-ESP participants as well. 

This instrument is currently being revised. The new version will 
be retitled and include forms for teachers, principals, parents and 
students. iSie reliability and validity data being collected is for the 
new instrument. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instruroent Tit le ; 
Academic Efficiency Index 

2. Author a t 

R. Narzano and C.L. Hutchins 

3 . I nstitution : 

Mid-continent Regional Educational Laboratory 

4. Format and Components : 

a) vorksheet for estimating out of class time use and absenteeism. 

b) observation instruments for determining classroom 
non-instructional and instructional time along with student 
engagement during Instructional time. 

5. Intended Purpose and Uses ; 

a) As a tool with which schools may assess their use of time in 
and out of class. 

b) As a staff development tool to be used within a coaching 
model* 

6. Reliability Information ; 

.85 inter-^rater reliability with one half-day training. 

7. Validation Information ; 
None available at present. 

8. Distribution Permission ; 
Limited. 

9 • Supplemental Materials : 

Training generally necessary for accurate use of the instrument. 
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MCR£L 



10. Content/DeBcription (Effective Schools Measures) ; 
Data regarding the following areas are first gathered: 

a) Time spent per day in scheduled non-instructional activities: 
e.g. lunch 

recess 

break between classes 
assemblies 

b) Average absentee rate per day 

c) Average time spent per day in non-instructional "in-class'* 
activities, 

e.g. beginning managerial activities 
ending managerial activities 
socializing 
disciplining 
interruptions 

d) Student engagement. 

e) Student success. 

These data are then combined to form indices which measure: 

a) Proportion of school day devoted to scheduling. 

b) Proportion of school day lost to absenteeism. 

c) Proportion of school log deviated to non-instructional in-class 
activities. 

d) Proportion of school day lost to student inattentiveness. 

e) Proportion of school day in which students are engaged* 

f) Proportion of school day lost to student lack of success. 

g) Proportion of school day devoted to academic learning time. 

1 1 . Comments (Effective Schools Measures) t 

Data is based on direct observation of instructional practices. 
The process itself requires that schools/teachers collect objective 
information about themselves. This appears useful as a staff 
development exercise . 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instrument Title ; 

Achievement Directed Leadership 
Engagement Rate Form 

2. Authors ; 
David Helms 

3. Institution ; ^ 
Research for Better Schools 

4. Format and Components ; 

Form to be completed by outside adult (not the classroom teacher 
for the class being observed) . Required training program. 

5. Intended Purpose and Uses : 

To measure engaged time, on task and off task behavior; for use in 
any content area. 

6. Reliability Information ; 
unknown. 

7. Validation Information; 



8. Distribution Permission ; 

McREL does not have this instrumentation. 

9. Supplemental Material s; 

1. Teachers' Guide Time Leaders' Guide 

2. Principals' Guide (Manual) 

3. District Guide 

4. videotape of classroom behavior/activity 
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Research for Better Schools 



10. Content/Description (Effective Schools Measures) : 

Codes each student in 5 categories of off-task behavior or 1 
category engaged, if a student is unengaged, a tally mark is made 
in the appropriate box on the form. No mark is made for engaged 
students. Scorable by the observer. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instrument Title : 
Indicators of Quality 

2. Authors : 

Vkn. Vincent & Jo Casey, 1968. 

3, Institution : 
Columbia University 

New York Institute of Administrative Research 

4, Format and Components : 
Observation Instrument-Guide 

5» Intended Purpose and Uses : 

To obtain a quantitative measure of scnool quality for use K-12. 

6, Reliability Information : 

Spearman Brcm\ reliability coefficient for total instrument .91. 
Split hal?.s reliability coefficient .84 for internal consistency. 

?• Validation Information ; 

Flanagan's tables from item analysis used to estimate 
discrimination indices in each of the 102 positive and negative 
signs. All but one discriminated in expected direction - ranged 
from +,65 - +.12 for positive item and +.61 - ,00 for negative 
signs. 

8, Distribution Permission ; 

McREL does not have the instrumentation. 

9. Supplemental Materials : 

1) ERIC document ED 034 308 

2) Manual 

3) Scoring and Report Service 
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Columbia University, N^Y. Institute of Admin. Research 



10. Content/Description (Effective Schools Measurea) ; 

Indicators of Quality is an instrument consisting of E1 polarized 
items, scorable on a 103-item scale applied to a representative 
sampling of all class meetings in a school* 3 instructional 
setting, itie procedure is observation of a uniform time span in 
each location, it requires a 3-day training and trial application 
period. 

The instrument is not available without training at development 
site. Scores are reported in relation to norms being developed and 
quality control charts (based on standard score scales) facilitate 
interpretation. 

The behaviors recorded include individualization, interpersonal 
regard, creativity, and group activity. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1. Instrument Title ? 

Checklist for Effective Schools 

2. Authors t 
Richard Hirsh 

3. Institution ; 

Reported in Instructor ^ 1982. 
University of Oregon. 

4. Format and Components ; 
Sample II item checklist. 

5. Intended Purpose and Uses ; 

Individual Building Levex Checklist for principals or teachers* 
use. 

6. Reliability Information ; 
Not Applicable. 

7. Validation Information : 

8. Distribution Permission ; 
Implied. 

9. Supplemental Materials ; 

Article; How Effective is Your School? 
Instructor , October , 1 982. 
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University of Oregon 

10* Content/Description (Effective Schools Measures) ; 
Checklist items content x 

1) clear academic and social goals 

2) curriculum linked to school-^^ide goals 
3} teachers check student progress 

4) everyone understands and accepts school rules 

5) student responsibility for learning and behavior 
6} promote high academic learning time 

7) teachers and principals care about students 

6) principals are strong leaders 
9) parent and couvnunity support 



1 1 . Comments (Effective Schools Measures) ; 

Best used as bulletin board reminder or introduction to school 
effectiveness research findings* Checklist is brief and 
incomplete. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instrument Title : 

Framework for Analysis of Your School Improvement Endeavor As It 
Relates to School Effectiveness Characteristics. 

2. Authors : 

Contact: Maureen McCormack-Larkin 
Project RISE 

3. Institution : 

Milwaukee Public Schools 

Division of Curriculum and Instruction 

Department of Elementary and Secondary Education 

4. Format and Components ; 

9 page questionnaire to be completed by principal (elementary 
level) . 

5. Intended Purpose and Uses : 

A means for evaluation of the effects of building level school 
effectiveness plans at the end of the year. Results to be used as 
a basis for revising 2nd year plans. Used in project RISE. 

6. Reliability Information : 
None 

7 . Validation Information ; 
None 

8. Distribution Permission : 
Contact Milwaukee Public Schools 

9 . Supplemental Materials ; 

1 ) Essential Elements of Effective Schooling on the Elementary 
Level. Used as basis for school effectiveness plans in project 
RISE schools (2 page handout) . 

2) Line graphs of performance on Metropolitan achievement tests. 
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Milwaukee Project AISE 

10. Content/Description (Effective Schcx^ls Measures) ; 

Itie project RISE questionnaire contains 67 items^ some of which are 
close-ended (yes or no) and some of whicn are open-ended. Two 
questions cover the school's academic mission; 2 questions cover high 
expectations; 6 questions cover student sense of affiliation? 5 
questions cover recognition of academic excellence; 3 cover school 
atmosphere; 6 cover grade-level objectives; 5 cover full content 
coverage; 3 cover accelerated curriculum; 9 cover structured learning 
environment; 6 cover key instructional behaviors; 2 cover direct 
instruction; 9 cover increased allocated time; 5 cover maximizing 
academic engaged time; 2 cover kindergarten instructional program; 5 
cover implementation of coordinated services (includes coverage of 
"pull out** procedures for Chapter I) or frequent evaluation of student 
progress; 6 cover implementation of evaluation component; 3 cover test 
taking or report cards; 7 questions cover implementation of the 
parent/community support component^ e.g.^ homework, attendance, and 
behavior policies. 

1 1 • Comments (Effective Schools Measures) : 

TSiis instrument would be extremely tedious to tally responses and 
time consuming. Approximately half of tne questions are open-ended. 
It was intended for in-building use and self-assessment, 
self-administered, its ♦■.ended use as an a posteriori measure does 
not make it feasible for i priori diagnostic tool. 
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EFFECTIVE SCHOOLS MEASURES 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instrument Title : 

Pupil Activities Record - Revised (PAR-R) ^ 

2. A uthors : 

Doss, Ligon and Friedman, 1980. 

3. Institution ; 

Austin Independent School District 
Office of Research and Evaluation 

4 . Format and Components ; 

Codes Time on Task and off task behavior within each basic skills 
subject area. 

5. Intended Purpose and Uses ; 

Rates pupil on task or off task behavior within each basic skills 
subject area. 

6. Reliability Information ; 

Interrater reliability coefficients for individual codes ranged 
from .00 - .99. Hie majority of coefficients were above .9000. 

7 . Validation Information : 

See Ross and Ligon Final Technical Report; 1980-81 ESEA Title I 
Regular Program Publication No . 80 . VT^ Aus t i n independent SchooT 
District, Office of Research and Evaluation, 1980. 

8. Distribution Permission : 

McREL does not have instrumentation or manual. 

9. Supplemental Materials : 

1) machine scorable 

2) manual available from Austin Independent School, Austin 
Publication #78.48. 
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Austin Independent School District 



'iO. Content/Description (Effective Schools Measures) : 

Itie system measures several context variables including pupil 
amount of time 8E>ent in each instructional area, size of 
instructional group, place of instruction, mode of instruction, 
content of instruction, contact with adults, teacher questioning 
strategy, teachers' use of a signal system and language of 
instruction (Qiglish, Spanish, etc.) • it focuses on observation of 
one pupil for one entire school day, using one minute intervals. 
Coder training requires approximately 5 days and consists of 2 days 
of instruction and 3 days of field practice. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instrument Title : 

Secondary Observation Instrument (SOI) 

2. Author s; 

Stallings, J., and Needels, M., 1981. 

3. Institution s 

SRI International • 

4. Format and Components ; 

3 sections: 1) on and off task student activities 

2) 5 minute interaction section 

3) classroom snapshot section 

5. Intended Purpose and Uses ; 

For use across all secondary grade levels, all academic subject 
areas. Used to record both teacher and student behavior. 

6 . Reliability Information ; 

Interobserver reliability coefficients for roost coding categories 
90 or above. 

7 . Validation Information ; 

See Instrument Training Manual. 

8 . Distribution Permission ; 

McREL does not have this instrument. 

9 . Supplemental Materials ; 

1. Secondary Observation Instrument Training Manual ; Menlo Park, 
Calif.; SRI International^ January^ 1981. 

2. precoded Videotape. 
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SRI International 

10. Content/Description {Effective Schools Measures) ; 

Instrument records codings of interactive on task activities (e.g., 
"Reading Along", "Discussion"), Non-interactive On-Task Activities 
(e.g., "Reading Silently") and Off-Task Activities (e.g., "Students 
Uninvolved") , it records context variables in the snapshot section 
with coding categories for activity, materials, number and kind of 
groupings, and if an adult is present. Records interactions in 
categories of "who", "to whom", "what" and "how". Identification 
and classroom information is coded once per classroom period. 
Classroom snapshot and 5 minute interaction are coded 5 times per 
classroom period. 

Observers can be trained to use the classroom snapshot in 2 hours. 
Training for the 5 minute interaction section required 5 days. At 
the end of the training, all observers must reach a criterion level 
of 80% or above agreement with a preceded videotape. 

1 1 . Comments (Effective Schools Measures) ; 

The SOI is unique with respect to its design for secondary use. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

Instrument Title : 

Student-level Observation of Beginning Reading (SOBR) 
Authors : 

Loraax, R.; Feinhardt, G. and Seewald, A.M.; 1980-81. 

Institution: 



Format and Components : 

Instrum.jnt records on and off task behavior of students and teache 
or aide behavior. 

Intended Purpose and Uses : 

Measure engaged learning time in 8 categories of student behavior 
and 5 categories of teacher and aide behavior and 2 categories of 
reading or math content. For use in all grade levels for coding 
any type of instruction that includes reading. 

Reliability Information : 

Average stability coefficient of .98 with 30 hrs. of observation 
per student and over .85 with 5 hours. Average 80% interobserver 
agreement. 

Validation Information ; 

See Lomaxr R. A Gener alizability Study of the Classroom 
Observations of Learning Disaoled Students. Paper presented at 
AERA, Boston, 1980. 

Distribution Permission : 

McREL does not have this instrumentation. 

Supplemental Materials : 

1. Self-instructional Training Manual 

2. Hand-scorable 
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Lomax 

10* Content/Description (Effective Schools Measures) ; 

Instrument records coding of on-task and off-task behavior, such as 
"waiting", "direct reading", "academic other", etc. Teacher and 
aide behavior codes include "academic other", "management", 
"reading instruction", and breakdown codes of "direct", and 
"indirect reading" « "Oxe time sampling interval is 20 seconds per 
child. Training requires approximately 15 hours and includes 5 
trainer-led sessions and field practice. 

1 1 . Comments (Effective Schools Measures) : 

Fbrmat is unique in coding affective behavior, classifying 
"positive" or "negative". 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 • Instrument Title : 

Student Engagement Ratings 
2. Authors ; 

Eramerf Evertson and Clements^ 1980* 
3* Institution ; 

University of Texas at Austin 

Research and Development Center for Teacher Education 
4 . Format and Components ; 

Engaged learning time observation system 

5 • Intended Purpose and Uses ; 

Whole class or small group format^ lesson topic and number of 
students coded; measures on and off-task behavior* 

6. Reliability Information ; 

Interobserver Reliability Coefficients reported; 
Definitely on task. Academic « .Tl 
Probably on task^ Academic * .0 
Definitely on task. Procedural » .67 
Probably on task. Procedural » .65 
Off taskr Sanctioned « .78 

Off task. Unsanctioned « .74 

Dead time « .0 

7. Validation Information : 

Emmer^ Evertson and Anderson obtained a correlation of .51 between 
number of students on task per observation in the first 3 weeks of 
school and number of students on task per observation - during 
remainder of school year. A correlation of .54 was obtained for 
similar comparisons of off-task behavior. 

8. Distribution Permission ; 

McREL does not possess these materials; Contact University of 
Texas at Austin Center. 

9 . Supplemental Materials ; 

1) Requires 2-3 hours of group instruction and practice trials 
using videotapes of classroom activity 

2) Training manual 
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. of Texas-Austin 



0. Content/Description (Effective Schools Measures) ; 

Engaged learning time variables measured include: 

1) definitely on task academic 

2) probably on task academic 

3) definitely on task procedural 

4) probably on task procedural 

5) off task sanctioned 

6) off task unsanctioned 
7} dead time 

8) no data 

It codes all students in the class and is applicable across all 
grades, all academic subject areas. TSie sampling interval varies 
from 4 ratings per hour (every 15 minutes) to 6 ratings per hour 
(every 10 minutes). Authors recommend coding 7^8 hours across 
several days in the first 3 weeks of school and 7-8 hours across 
several days for the remainder of the school year. 
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EFFECTIVE SCHOOLS MEASURES 
DESCRIPTION AND CRITIQUE REPORT 

1 . Instrument Title ; 

Beginning Teacher Evaluation Study (BTES) 
Observation System 

2. Authors ; 

Marbaire, R. , Fisher, C.W., Filby, N.N., and Dishaw, M. , 1976. 

3. Institution s 

Far West Laboratory for Educational Research and Development. 

4. Format and Components ; 

Codes behavior in categories representing setting, content of 
instruction, difficulty level of work, instructor behavior and 
interactions with students. 

5. Intended Purpose and Uses ; 

Instrument measures engaged learning time for use in reading and 
mathematics classes; was used in second and fifth grade 
classrooms. 

6. Reliability Information ; 

Inter-observer Reliability Coefficients for individual codes ranged 
from .07 - .99; majority above .80. Coefficients representing 
stability of coding categories across observers and occasions for 
task difficulty categories ranged from .21 to .80, with majority 
above .50. 

7 . Validation Information ; 

See Technical Report 1-5, Beginning Teacher Evaluation Study , 1977, 
available from Far West Lab. 

8 . Distribution Permission ; 

Contact authors: McREL does not have this instrumentation. 

9. Supplemental Materials; 



1 . Manual 

2. Machine scorable forms 
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Far West Laboratory for Bduc. Research and Dev. 

10. Content/Description (Effective Schools Measures) ; 

Instrument records codings of content and context variables 
including: 1) general content [wait, transition, management, 
break, non-academic instruction, academic instruction] , 2) setting 
[self-paced, other-paced] , 3) instructor moves [academic 
observational monitoring, academic feedback, question, explanationr 
etc.], 4) focus of instructor move [teacher, adult, etc.], 5) 
reading content [decoding, word structure, etc.]f 6) math content 
[division, multiplication, etc.], The observ r judges the 
student •s success rate as easy, medium or difficult • Target 
students are observed md their behavior recorded in 3-6 minute 
cycles • 

11. Comments (Effective S *hools Measures) : 

T^is is one of few instruments in which judgement is made and coded 
concerning students* success rates • Instrumentation system would 
need modifications in content categories to be applicable across 
all grade levels. 
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EFFECTIVE SCHOOLS MEASURES - 
DESCRIPTION AND CRITIQUE REPORT 

1 • Instrument Title ; 

Good and Becker man System 

2, Authors : 

Good, T*L. and Beckerman, T.M., 1978. 

3, Institution : 

Research for Better Schools 

4, Format and Components ; 
Hand scorable 

5* Intended Purpose and Uses ; 

Measures engaged learning time 

1) student involvement type, 2) setting type, 3) activity type. To 
be used across all grades, all subject areas. 

6. Reliability Information ; 

Interrater reliability/agreement of 85% (among 6 coders) . 

7. Validation Information ; 

8. Distribution Permission : 

Contact Good and Beckerman. McREL does not have this 
instrumentation. 

9 . Supplemental Materials ; 

i 

Training requires approximately 20 hours, consisting of 
instruction, coding sessions with videotapes and field practice. 
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Research Cor Better Schools 

10* Content/Description (Effective Schools Measures) : 

Instrument records behavior of all schools in a class; the observer 
codes one pupil and moves on to the next. All pupils are coded in 
5 minute cycles. 
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LIST OF PROGRAMS AND DISTRICTS 
Developer and program title Elem MS/JHS SHS 

aska 



Department of Education, 

Effective Schooling Project 

kansas 



State Education Agency, 

Program for Effective Teaching 

ulifornia 



DK 



1 



DK 



DK 



Districts Using 



Alaska Gateway; AK 
Anchorage# AK 
Mat-Sur .iK 



Approx. 470 Districts 



Los Angeles Oounty Schools 
. Quality Skill Building Program 

9 

San Mateo Public Schools, 
Untitled 



435 



75 



100 lios Angeles, CA 



San Mateo, CA 



Stallings Teaching and Learning Instit* DK 
Effective Use of TiiTie Program 



olorado 



DK 



DK Cuper*:ino, CA 

Mountain View/Los Altos 
Sun'-yvale, CA 
Wisman, CA 
Washington, DC 
Detroit, MI 
Putnam County, WV 



Department of Education, 

Accountability/ Accreditation 
Prog /am 

Department of Education, 
School Climate Program 

Loveland Public Schools, 
Untitled 



DK 



DK 



15 



DK Approx, 90 Districts 



15 Approx. 60 Districts 



Loveland, CO 



Mid-continent Regional Educational 
Laboratory, 

Effective Schools Program 

Denver Puolic Schools, 

Interact i and Achievement 



DK 



DK 



DK 



Approx 40 Districts 



Denver, 00 



This list is organized by state of the program's developer. Many programs are being 

implemented outside their state of origin. 
•K - respondent does not know. 

Courtesy of Matthew B. Miles, Center for Policy Research, Inc.: Updated 3/84) 
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>- 
a. 
o 

I— 

UJ 



Connecticut 

New Haven Public Schools, 
Urban Academy 

State Department of Education, 
School Effectiveness Project 



Delaware 

State Department of Public 
Instruction, 
Untitled 



Kentucky 

State Departfnent of Education, 
School Climate Improvement 



Maryland 

State nof>artn->f^nt of Education 
Srfioot I:nprovenHMit through 
Instruc t ional Improvernera (SITIP) 



28 

21 
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New Haven, CT 



Coventry, CT 
Griswoid, CT 
Hartford, CT 
Meriden, CT 
New Britain, CT 
New London, CT 
Oxford, CT 
Stamford, CT 
Vernon, CT 
West Haven, CT 



^0 26 Appoquinimink, DE 
Brandywine, DE 
Caesar Rodney, DE 
Cape Henlopen, DE 
Capital, DE 
Christina, DE 
Colonial, DE 
Delnnar, DE 
Indian River, DE 
Lake Forest, DE 
Laurel, DE 
Milford, DE 
P<?d Cb>, DE 
Seaxord, DE 
Smyrna, DE 
Woodbridge, DE 



Buckner, KY 
Cannpbellsville, KY 
Clinton, KY 
Danville, KY 
Ehzabothtown, KY 
Floret^rc, KY 
Frankfort, KY 
LouiSviUo, KY 
Rit hftiond, K \ 



All 2^ counties 



Ortroit f\jt)li« S( lu^ol',, 

Sc tiool l:ii|)rv^v(MTUMU IV()>',riifTi 



8 



Ootroil, MI 
Detroit, Ml 
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BEST COPY Wftll ABLE 



Michigan cont'd 

Michigan MiddleXities Association, 
Untitled 



- 3 - 

DK 



Missouri 

State Department of Education, 

Instructional Management System 

New Jersey 

Department of Education 

Comprehensive Basic Skills Review 



DK 



67 

DK DK Battle Creek, MI 
Bay City, MI 
Benton Harbor, MI 
Flint, Ml 
Grand Rapids, MI 
Jackson, MI 
Kalamazoo, MI 
Lansing, MI 
Marquette, MI 
Midland, MI 
Monroe, MI 
Muskegon, MI 
Muskegon Heights, MI 
Niles, MI 
Pontiac, MI 
Saginaw, MI 
Southfield, MI 
Willow Run, MI 
Ypsilanti, MI 



DK DK Approx, 200 districts 
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12 



Ncv.-ark Piiblir ^.rfiools, ^ 
Tntitlod (based on study, Characteristics 
of Hi>;b Arhiovmg Elementary Sct^ools 



0 



Mexico 

hrpar tmcf^t of .lust i<^e. 

School ("lunate Iirj[)rOvefnent 



17 



20 



30 (Secondary users only) 
Pleasantville, N3 
Camden, N3 
Newark, N3 
East Orange, N3 
Orange, N3 
Irvington, N3 
Sussex, N3 
3ersev City, N3 
Hoboker., N3 
Trenton, N3 
New Brunswick, N3 
Keansburg, N3 
Paterson, N3 
Passaic, N3 

0 Newark, N3 



26 districts 
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BEST COPY AVA 



( » A 
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New York 

New York City Schools, 

High School Improvement Project 

New York City Schools, 

School Improvement Program 

New York Urban Coalition, 

Local School Development Program 

North Carolina 

Center for Early Adolescence, 

Middle Grades Assessment Program 



68 



Ohio 



Department of Education, 

and Kent State Center for Educational 
Development and Strategic Services 
(KEDS) 



Orc\\on 

Nor!h\Kest Regional Educational 
Laboratory, 
CiOal-Basod Lducation 



Portland {"'ubiii Schools, 

Studom Achievcinont Policy 



25 



37 



18 



100 



0 

0 

13 



10 



New York, NY 
New York, NY 
New York, NY 



Charlotte City, NC 
Durham, NC 
Greensboro, NC 
Johnston City, NC 
St. Louis, MO 
Pocantico Hills, NY 
East Cleveland, OH 



10 Bedford, OH 
Canton, OH 
Cincinnati, OH 
Cleveland, OH 
Dayton, OH 
Columbus, OH 
East Cleveland, OH 
Greenhills-Forest Park, OH 
Lorain, OH 
Mansfield, OH 
Shaker Heights, OH 
Stevibcnville, OH 
Streetsboro, OH 
Tallmadge, OH 
Toledo. OH 

Warrensville Heights, OH 
\ ounj;stown, OH 



I 3 Camus, \V ^ 
Seattlo, WA 
Taroina, NX' -X 
Yakima, WA 

Sec also Alaska SEA (3 districts) 
10 Portland. OR 



BEST COPY AVAILABLE 



ERJ.C 



69 



Pennsylvania 

Philadelphia Public Schools, 
Expectations Project 

Research for Better Schools 

and New Jersey Education Association, 
School Effectiveness Training 



Research for Better Schools and New 
Jersey Education Association, 
School Effectiveness Training 
(High School Version) 

Research for Better Schools, 

Achievement-Directed Leadership 

Utah 

Ogden Public Schools, 

incentive-Productivity Model 

Salt Lake City Schools, 
School Climate Program 

Vermont 

University of Vermont, 
Untitled 



0 



15 



0 
2 



0 
3 



0 



0 
0 



0 

1 



Philadelphia, PA 



Atlantic City, N3 
Jersey City, N3 
Paterson, N3 
Camden, N3 
Piainfield, NJ 
Reading, PA 

Atlantic City, N3 



Appoquinimink, DE 
New Brunswick, NJ 
Bethlehem, PA 



Ogden, UT 



Salt Lake City, UT 



Cabot, VT 
Duxbury, VT 
Hardwick, VT 



Seattle Public Schools, 

School Effectiveness Project 

Seattle l\jbli«- Schools, 
Projoct ACCESS 

VI I'^v'onsin 

L'n;vcrsitv of Vtisconsin. 

W ist onsiti Program for the 
Ivcni>vval and Improvement of 
Socondary Education (WHISE) 



0 



BEST COPY AVWL^BLE 



0 



i<0 



10 



20 



Seattle, WA 
Seattle, WA 



Appleton, WI 
Barneveld, U I 
Balsam Lake, WI 
Beloit, WI 
Brown Deer, WI 
C'ameron, WI 
East Troy, WI 
Franklin, WI 
Green Ray, WI 
Hartford, WI 
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Wisconsin cont'd 



70 



Milwaukee Public Schools, 
Project RISE (Rising to 
Individual School Excellence) 



18 



Hartlanc, WI 
Ouda, WI 
Keshena, WI 
Maple, WI 
Mequon, WI 
Milton, WI 
Oconto Falls, WI 
Plymouth, WI 
Port Edwards, WI 
Portage, WI 
Prairie du Sac, WI 
Rhinelander, WI 
Seymour, WI 
Sheboygan Falls, WI 
Watertown, WI 
Waunakee, WI 
Wauwatosa, WI 
Whitefish Bay, WI 

Milwaukee, WI 
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Section III 

Annotated Bibliography of Related Instruments 

Pox, R.W. ••How Positive is Your School Climate?" in "School Climate 

Improvement; A Challenge to the School Administrator." Phi Delta 
Kappan , October, 1982. 

This is a ••checkUst" designed to give a snapshot of school 
climate. Ten items are rank ordered from "^crongly disagree" to 
"strongly agree." 

Hathaway, W. Student Achievement Policy . Portland, Oregon schools, 
n.d. 

This document includes a needs assessment checklist called 
"Instructional Effectiveness Checklist" of 15 items on 
instructional practices. It can be used by both elementary and 
secondary teachers. 

Herskowitz, H. and Paynther, T. "Procedural Guide for Implementing 
Effective Schools Characteristics in a Desegregated Setting." 
Kent State Center for Educational Development. 
The appendix of this document contains sample items from the 
Connecticut School Effectiveness Questionnaire and Interview. 
Also included is a Leader •s Study Guide for Videotape, 
Desegregation and Improving School Effectiveness , 1981. 

Howard, Eugene. "Are You a Good Climate Leader?" in Howard, Classroom 
Control Index , School Discipline Desk Book . West Nyack, NYr 
Parker Publishing Co., 1978. 

A 15 item questionnaire that assesses relationships with students, 
managing the classroom, and instructional techniques. Response 
categories range from "almost never" to "almost always." 

Howard, Eugene. Successful Practices for Making the Curriculum More 
Flexible . Denver, Colorado; copyright 1982. 
This instrument consists of a list of practices reported as 
promising or successful in schools in which curriculum flexibility 
is desired. It rests in the assumption that the curriculum can be 
varied to accommodate individual differences. Responses are rated 
in 2 categories, "What Is" and "Impact Potential." 

Howard, Eugene. Mini Audit #2. Activities and Projects for Climate 
Improvement . Process Determinants and Material Determinants , 
n ,d • 

The system includes a packet containing the mini audit measure. 
T\\is consists of a 9-page questionnaire {55 items) designed to 
assess the climate of a school in terms of specific activities, 
programs and practices. Respondents (teacher, parent, 
administrator) rank order items in two response categories, "What 
Is" and "What Should Be." 

Klausmeier, H., Lipham, J. and Daresh, J. The Renewal and Improve ment 
o f Secondary Education; Concepts and Practices . University of 
Wisconsin; Research and Development Center for Individualized 
Schooling, 1980. 
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This document contains a series of questionnaires including a 
desirability scale of comprehensive objectives, a Learning Stv 'es^ 
Inventory (for students) and a form for recording amount of tune 
allocated per course in a student course rating form. Other 
staff, students' and professors' questionnaires (some pre-post 
text) are included for evaluation of field test (RISE Program) . 

New York Urban Coalition. Local school development project. 

This document includes in the appendix a checklist for school 
improvement planning team members and a staff questionnaire. The 
questionnaire is rated on a Likert-type scale in 2 response 
categories, "What Is" and "What Should Be." It assesses 
perceptions regarding goals, student assessment, instructional 
methods, materials, coordination, resources, and time and 
scheduling. A separate math assessment, reading assessment and 
language skills development perceptions assessment, and . -^hool 
environment measure, is incluued. The student and parent 
questionnaires assess attitudes. 

Nordengren, C, Veseth, R., McDowell, K. and Lindahl, R. New Mexico 
School Climate Improvement Process . 1973. 

Report of a project supported by a grant from the Office of 
Juvenile Justice and Delinquency Prevention, U*S. Department of 
Justice. The in?>rovement prograun consists of 8 steps: 

1) Raising the level of faculty, student and parent awareness 
(through workshops) . 

2) Forming School Climate Improvement Committee (man ^es, 
supports, and assesses) • 

3) Collecting baseline data (on morale and climate s^. ^vtoms) . 

4) Assessing the School's Climate (a mini-audit by visiting 
team) • 

5) Brainstorming and prioritizing (judgements based on 
mini-audit) , 

6) Forming task forces (to initiate projects and activities) . 

7) Managing task forces (by SCIC and the principal). 

8) Evaluating (instruments and procedures to evaluate the extent 
of school climate improvement) . 

The instrument is the CFK Ltd. School Climate Profile , a 
Likert-type, 5-page questionnaire requiring two responses per 
item, "What Is" and "What Should Be." The questionnaire can be 
completed by any staff member or student. It assesses perceptions 
regarding such cateyories as trust, respect, morale, cohesiveness , 
school renewal, input opportunity, academic and social growth and 
caring. 

Ohio Department of Education. The Ohio Academy for School Improvement 
S trategies; Proceedings of a Conference Hel d_ O n August 2-6, 1982 
at the Ohio State University . Columbus, OhI iPublished by Ohio 
State Department of Education, 1982. 

This document includes a Summary of Pre-Oasis Needs Assessment . 
Items are ranked as 2 categor ies : 1) General Needs, and 2) My 
Personal Needs. Items reflect skills to improve instruction, 
specialized knowledge and curriculum projects; 3) Basic 
Instruction, and 4) Teaching/Learning Model. 
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Seattle Public Schools, Burton, N., Toews, E. and Binhaum, D. 

School-based Planning Manual . Project ACCESS Report funded by the 
Pord Poundat ion, 1982. 

Part II, Supplementary Materials, includes a 30-itero needs 
assessment covering ••improved Academic Achievement" and "School 
Climate." It can be completed by staff, student or citizen. 
Part I is a planning guide. 

Sloan, E. Y ear-end Progress Report: "Expectations Project" . Submitted 
to Edward McHazen Foundation by School DistrTct of Philadelphia, 
1979. 

This document includes a series of many questionnaires and 
observation guides that reflect on school assessment, the 
community perceptions, staff beliefs, administrator practices, 
etc. (including "Use of the P.A. System") and discussion questions 
for feeding information back to staff. Some observation guides 
use a cloze format. It lists some interesting and unusual 
questions and observation formats. 
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Edmonds, R. Search for Effective Schoolst The Identification and 
Analysis of City Schools that are Instructionally Effective for 
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Preder ickson, J. Models for Determining a School's Effectiveness . 
1980, paper presented at AERA, Boston. ERIC. 
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Section V 
Instruments in Prepamtion and Dates 

1) Cleveland School Effectiveness Project - Evaluation instrumentj 
being developed. Contact: Maureen McCormack-Lar kin, Milwaukee 
Wise. n.d. 

2) Achievement Directed Leadership Program. 
Contact: David Helms Dec. *83 

Developing Instruments and materials sent to NIE. 

3) Chapter I 

Contact: Skip McCann 

Comprehensive basic skills review process attending to 15 
classroom/school variables. Instcoraentation a^'ailable Dec. '83 

4) Urban Education Component 
The Regional Exchange - rbS 
Contact: Ton\ Corcoran 
Instrumentation available Dec. '83. 



